Method and system for obtaining a first signal for analysis to characterize at least one periodic component thereof

ABSTRACT

A method of facilitating obtaining a first signal, for analysis to characterize at least one periodic component, includes obtaining two second signals representative of intensities of electromagnetic radiation. The first signal is at least derivable from an output signal obtainable by applying a transformation to the second signals such that any value of the output signal is based on values from each respective second signal at corresponding points in time. The method further includes obtaining a value of a variable determining influences of components of respective second signals on the output signal when the signals corresponding to the second signals are captured and the transformation is applied, by (i) analyzing the first, second and/or the output signals to select a value of a parameter corresponding to a respective one of the variables; or (ii) calculating values of at least one time-varying factor corresponding to a respective one of the variables.

This is a continuation of prior U.S. application Ser. No. 14/172,591,filed Feb. 4, 2014, which is a continuation of prior U.S. patentapplication Ser. No. 13/500,000, filed Apr. 3, 2012, now U.S. Pat. No.8,666,116, issued Mar. 4, 2014, which is a National Stage application ofPCT/IB2010/054396, filed Sep. 29, 2010, and which claims the benefit ofEuropean Patent Application No. 09172337.9, filed Oct. 6, 2009, theentire contents of each of which are incorporated herein by referencethereto.

The invention relates to a method of facilitating obtaining a firstsignal for analysis to characterize at least one periodic componentthereof.

The invention also relates to a system for obtaining a first signal foranalysis to characterize at least one periodic component thereof.

The invention also relates to a computer program.

European patent application No. 09154493.2, in the name of the sameapplicant as the present application, was filed before and publishedafter the priority date of the present application. It describes asystem arranged to process a sequence of images. The system carries outthis processing in order to determine at least one of the presence and afrequency value of at least one peak in the spectrum of a signal basedon the pixel data of the images corresponding to a frequency of aperiodic physiological phenomenon. An image segmentation algorithm isperformed on at least one of the sequence of images. One or moredistinct segments determined to correspond to a body part of a desiredtype are tracked through the sequence of images. For each selected andtracked segment a measurement zone is selected. For each measurementzone, a signal representative of the time-varying average brightness ofthe pixels corresponding to the measurement zone is generated. Thebrightness may be a weighted sum of the color components or only thevalue of one color component. The signal is centered on its mean value,and basic signal processing techniques are used to determine one or morelocal maxima of the spectrum of the signal, at least within a rangeknown to comprise typical heart rate values and/or respiration ratevalues for the living beings of interest (generally humans).

It is desirable to provide a method, system and computer program forfacilitating obtaining a first signal for analysis to characterize atleast one periodic component thereof in which the components of interestare relatively clearly distinguished from noise, for example noise dueto motion and illumination changes.

To this end, the invention proposes a method of facilitating obtaining afirst signal for analysis to characterize at least one periodiccomponent thereof, including:

obtaining at least two second signals representative of intensities ofcaptured electromagnetic radiation, each corresponding to a respectivedifferent radiation frequency range,

the first signal being at least derivable from an output signalobtainable by applying a transformation to the second signals such thatany value of the output signal is based on values from each respectivesecond signal at corresponding points in time, the method furtherincluding:

obtaining at least one value of at least one variable determininginfluences of at least components of respective second signal on theoutput signal when the signals corresponding to the second signals arecaptured and the transformation is applied, by at least one of:

(i) analyzing at least one of the second signals, an output signalobtained by applying the transformation to the second signals and afirst signal derived from the output signal and

using the analysis to select at least one value of at least oneparameter corresponding to a respective one of the variables; and

(ii) calculating values of at least one time-varying factorcorresponding to a respective one of the variables, each factor valuebased on at least one second signal value,

and applying each factor in an operation in at least one of a number ofparallel sequences of operations comprising at least one such operationand taking a signal corresponding to a respective one of the secondsignals as input.

The method is suitable for facilitating a method in which reflected,transmitted or radiated light is captured and processed to obtain asignal representative of at least variations in a value based onintensity values in the different frequency ranges, also referred tohere as color channels. This signal is analyzed to characterize certainperiodic phenomena in the scene from which the reflected, transmittedand/or emitted light is captured. Because certain color channels exhibitweaker variations due to the phenomenon of interest than others, whereasmotion and illumination changes influence each of the second signals,the transformation applied to the second signals is suitable forobtaining an output signal with an increased signal-to-noise ratio.However, simply subtracting one of the second signals from another willgenerally not be sufficient to remove the influence of motion andillumination changes. By analyzing at least one of the second signals,the output signal and the first signal and using the analysis to selectat least one value of at least one parameter determining influences ofat least components of respective second signals on the output signalwhen the signals corresponding to the second signals are captured andthe transformation is applied, it is established whether and to whatextent each color channel should influence the output signal and thusthe first signal. The respective parameter values affect fewer than allof the second signals that are used to obtain the output signal. Theymay in particular affect the extent to which the second signalsinfluence the output signal, i.e. the relative strength of the secondsignals affected by the parameter values in the mix of second signalsused to obtain the output signal. Compared to methods that subtract anintensity signal obtained at one wavelength from one obtained atanother, or to those that take a ratio, the present method works wellwith second signals that each contain a component associated with theperiodic phenomenon of interest to at least a limited extent. Therelevant information from multiple channels is used, whilst at the sametime unwanted information is suppressed.

This is also the case where values of at least one time-varying factorare calculated and each factor value is based on at least one secondsignal value, because a kind or normalization is achieved. By applyingthe factors to signals based on respective ones of the second signals, atransformation, in particular a kind of normalization, in color space isobtained. Because the factor is time-varying, changes in the backgroundillumination spectrum can be removed, improving the signal-to-noiseratio. The signals obtained by applying the factors can be combinedsubsequently into one signal from which the first signal is at leastderivable, which is implied by the fact that the factors are variablesdetermining influences of at least components of respective secondsignals on the output signal when the transformation is applied. In thisrespect, the method differs from those that simply take a ratio of colorcomponents and characterize a periodic component of that ratio. It isobserved that the signals based on respective ones of the second signalseach correspond to a respective one of the second signals, a scalarmultiple thereof, or a signal obtainable by applying another of thefactors to one of the second signals. Thus, where the values of at leastone time-varying factor are calculated, each second signal forms theinput to a respective sequence of one or more operations in which thecalculated factors are applied. The outputs are then combined into anoutput signal from which the first signal is at least derivable.

In an embodiment, the analysis of at least one of the second signals, anoutput signal obtained by applying the transformation to the secondsignals and a first signal derived from the output signal is used toselect at least one value of at least one parameter determininginfluences of at least components of respective second signals on theoutput signal so as to maximize a relative signal strength in a limitedpart of a spectrum of the first signal.

Thus, where it is known a priori that a particular range of the totalspectrum of the first signal comprises the information associated withthe phenomenon of interest, the method improves the signal-to-noiseratio in that range.

In an embodiment, the transformation includes at least one parameterizedoperation and the analysis of at least one of the second signals, anoutput signal obtained by applying the transformation to the secondsignals and a first signal derived from the output signal is used toselect values of at least one parameter of the operation.

Compared to selecting parameters affecting the illumination of a scene,this embodiment is relatively certain to lead to good results. Moreover,it is less obtrusive than changing the lighting conditions.

In a variant, the operation is an operation taking at least two inputsbased in different ways on the second signals.

The inputs can simply correspond to values comprised in differentrespective second signals, or one can be a sum and the other adifference, for example. This embodiment enables the two inputs to becombined in such a way that the parameter values determine, for example,multiplication factors or phase changes applied to the inputs beingcombined.

In a further variant, the transformation includes an operation carriedout in parallel on respective input signals using at least one parametervalue specific to fewer than all of the input signals.

Thus, an alternative way of affecting the prominence of the differentsecond signals in the output signal is provided.

In a further variant, the transformation includes at least oneparameterized non-linear operation and the analysis of at least one ofthe second signals, an output signal obtained by applying thetransformation to the second signals and a first signal derived from theoutput signal is used to select values of at least one parameter of theoperation.

Thus, the influence of a particular color channel on the output signalof the transformation can be made dependent on amplitude and/orfrequency. This variant can, for example, be used to allow large-signalfluctuations in one color channel to influence the output signal but notsmall-signal fluctuations, or vice versa.

In an embodiment, the transformation includes an operation carried outin parallel on respective input signals, each a different linearcombination of second signals with at least one coefficient differingfrom zero, the operation comprising taking a logarithm of the inputsignal.

Generally, the operation will be carried out on respective input signalsthat are scalar multiples of the second signals or correspond to thesecond signals.

Variations in ambient illumination levels will in many situations affecteach color channel to approximately the same degree. In essence, eachsecond signal is the result of a modulation by a signal representativeof variations in ambient illumination levels. The output of thelogarithm is a signal that has an additive component related to themultiplicative variations in ambient illumination levels. One can thinkof the output as the sum of a signal representative of variations inambient illumination levels and a signal that is independent thereof andincludes a component representative of a periodic phenomenon ofinterest. The latter signal is generally different for each colorchannel, because each color channel is affected differently by theperiodic phenomenon. The signal representative of variations in ambientillumination levels can be suppressed relatively easily. One way ofdoing this is by means of a subsequent operation projecting onto a planein a multidimensional space input signals at least derived fromrespective output signals of the operation that comprises the taking ofthe logarithm. This multidimensional space will generally have a numberof dimensions equal to the number of second signals (e.g. three, wherethe second signals represent red, green and blue color channels). Themultidimensional space can be thought of as a color space. It is notedthat variations due to movement of a region of interest that isimperfectly tracked will result in similar effects as variations inambient illumination levels, which effects are similarly suppressed inthis embodiment.

In an embodiment, each second signal comprises a sequence of imageframes comprised of pixel values representative of intensities ofcaptured electromagnetic radiation in an associated radiation frequencyrange.

This embodiment is relatively easy to implement using a video camera,because such a camera already comprises filters to provide image framesin multiple color channels. Moreover, random noise can be reduced bycombining pixel values from different locations in the image to form thefirst signal or a precursor thereof.

In an embodiment, the second signals include a second signal comprisingintensity values of captured electromagnetic radiation in a part of theelectromagnetic spectrum tuned to a peak in an absorption spectrum ofwater.

This embodiment is suitable for facilitating obtaining a first signalfor analysis to characterize at least one periodic component thereofcorresponding to a biological phenomenon in a living being from whomradiated, reflected or transmitted light is captured. In particular,this variant can be used to facilitate the implementation of aphotoplethysmographic method for determining the heart rate of theliving being.

In an embodiment, the second signals include a second signal comprisingintensity values of captured electromagnetic radiation in a part of theelectromagnetic spectrum corresponding to a range of wavelength valuesbetween 500 nm and 600 nm.

This embodiment is similarly suitable for facilitating obtaining a firstsignal for analysis to characterize at least one periodic componentthereof corresponding to a biological phenomenon in a living being fromwhom radiated, reflected or transmitted light is captured. The secondsignal comprising intensity values of captured electromagnetic radiationin a part of the electromagnetic spectrum corresponding to a range ofwavelength values between 500 nm and 600 nm will contain informationabout changes in the level of oxyhemoglobin in illuminated (skin)tissue.

In an embodiment, the analysis includes carrying out a principalcomponent analysis on data sets respectively based on a plurality ofsecond signals.

This embodiment provides a suitable implementation of the analysis stepof the general method where one second signal is relatively stronglycorrelated with the periodic phenomenon of interest and the other signalrelatively weakly. The signals can be subjected to a matrix operation toobtain an output signal on which the first signal is based, inparticular to which it corresponds. The parameters of the matrixoperation are derived from the principal components. Carrying out aprincipal component analysis on the second signals allows one todetermine the correlations between the second signals. One of theprincipal components will correspond to the signal componentrepresentative of the periodic phenomenon of interest. The matrixoperation including parameters based on the principal component analysisis set up such as to retrieve this principal component as the firstsignal or to retrieve a signal from which the first signal is directlyobtainable.

An embodiment, wherein at least one of the second signals, an outputsignal obtained by applying the transformation to the second signals anda first signal derived from the output signal is analyzed and theanalysis is used to select at least one value of at least one parameterdetermining influences of at least components or respective secondsignals on the output signal when the signals corresponding to thesecond signals are captured and the transformation is applied, furtherincludes causing the selected values of the at least one parameter to beloaded into a system comprising:

an interface for obtaining at least two second signals representative ofintensities of captured electromagnetic radiation, each corresponding toa respective different radiation frequency range; and

a signal processing system for applying a transformation to the secondsignals to obtain an output signal from which the first signal is atleast derivable, any value of the output signal being based on valuesfrom each respective second signal at a common point in time,

wherein the system is arranged to cause influences of respective secondsignals on the output signal to be determined at least partly inaccordance with the loaded values of the at least one parameter.

This embodiment is thus used to find appropriate parameter values foruse in systems manufactured to carry out a method of obtaining the firstsignal by obtaining at least two second signals representative ofintensities of captured electromagnetic radiation, each corresponding toa respective different radiation frequency range, and applying thetransformation to the second signals produces an output signal of whichany value is based on values from each respective second signal atcorresponding points in time.

In an embodiment, the analysis and the selection of at least oneparameter value is carried out continually whilst processing the secondsignals to obtain the first signal.

Thus, this embodiment provides a method that continually adapts toensure that a first signal is provided in which the periodic componentof interest is relatively clearly defined.

In an embodiment, the at least one time-varying factor includes a factorobtainable by calculating a linear combination of corresponding valuesfrom each of at least one of the second signals, and the same value ofthat factor is used in each of the parallel sequences of operations.

An effect of this embodiment is to carry out a normalization withrespect to illumination variations unrelated to the periodic phenomenonof interest. Because the signals in the different color channels (thesecond signals) are affected to essentially the same degree by suchvariations, any linear combination will result in at least a degree ofremoval of the modulation due to the illumination variations.

In a variant of this embodiment, the parameters corresponding to arespective one of the variables determining influences of at leastcomponents of respective second signals on the output signal when thesignals corresponding to the second signals are captured and thetransformation is applied include at least one coefficient of the linearcombination.

In this embodiment, the factor can be given a value that results in atransformation to a plane in color space that is generally parallel to aline in color space that corresponds to variations due to the periodicphenomenon of interest. For example, in the case of remotephotoplethysmography, pulsating blood flow due to the heartbeat and/orrespiration of a subject represented in a sequence of color images willcause variations in the color of exposed skin of the subject. In theabsence of noise due to other causes of color changes in the images,these variations are along a line in color space. The projection onto aparallel plane will tend to remove the variations due to the othercauses.

In a variant in which each second signal comprises a sequence of imageframes comprised of pixel values representative of intensities ofcaptured electromagnetic radiation in an associated frequency range, theoperations in the parallel sequences are carried out per pixel, with afactor value calculated for each pixel position.

In another embodiment of the method, for at least one of the factors,separate values at corresponding associated points in time arecalculated for each of the parallel sequences, each obtained frommultiple values of the second signal taken as input by the sequenceconcerned, the multiple values being associated with respective pointsin time spanning an interval including the point in time associated withthe factor value.

An effect is to carry out a color normalization. Because the factorvalues are each obtained from multiple values of the second signal takenas input by the second sequence concerned, the multiple values beingassociated with respective points in time spanning an interval includingthe point in time associated with the factor value, the relativelyslowly varying background color changes can be removed. Those associatedwith the phenomenon of interest, e.g. pulsating blood flow of a subjectrepresented in image signals corresponding to the second signals,remain.

In a variant of this embodiment, calculating the factor value includescalculating an average of second signal values associated with at leasttwo points in time.

Averaging removes the influence of variations.

In a further variant, the second signals are discrete in time, and thecalculation of each factor value includes taking an average of secondsignal values associated with at least two points in time, of which onecorresponds to a point in time associated with the factor value.

The point in time associated with the factor value is the point in timeassociated with the signal value to which factor value is applied in therelevant operation in the sequence of operations. An effect of thisvariant is to ensure that the average as closely approximates aninstantaneous average value as possible, so that faster backgroundvariations can be removed more accurately.

According to another aspect, the system for obtaining a first signal foranalysis to characterize at least one periodic component thereofaccording to the invention includes: an interface for obtaining at leasttwo second signals representative of intensities of capturedelectromagnetic radiation, each corresponding to a respective differentradiation frequency range,

the first signal being at least derivable from an output signalobtainable by applying a transformation to the second signals such thatany value of the output signal is based on values from each respectivesecond signal at corresponding points in time, wherein the system isarranged to

obtain at least one value of at least one variable determininginfluences of at least components of respective second signals on theoutput signal when the signals corresponding to the second signals arecaptured and the transformation is applied, by at least one of:

(i) analyzing at least one of the second signals, the output signal andthe first signal and using the analysis to select at least one value ofat least one parameter corresponding to a respective one of thevariables; and

(ii) calculating values of at least one time-varying factorcorresponding to a respective one of the variables, each factor valuebase on at least one second signal value,

and applying each factor in an operation in at least one of a number ofparallel sequences of operations comprising at least one such operationand taking a signal corresponding to a respective one of the secondsignals as input.

In an embodiment, the system is configured to carry out a methodaccording to the invention.

According to another aspect of the invention, there is provided acomputer program including a set of instructions capable, whenincorporated in a machine-readable medium, of causing a system havinginformation processing capabilities to perform a method according to theinvention.

The invention will be explained in further detail with reference to theaccompanying drawings, in which:

FIG. 1 is a block diagram of a system for extracting information from asignal obtained by applying a transformation to at least two secondsignals representative of intensities of captured electromagneticradiation, each corresponding to a respective different radiationfrequency range;

FIG. 2 is a flow chart of a method carried out by the system;

FIG. 3 is a diagram illustrating the transformation applied by thesystem;

FIG. 4 is a flow chart illustrating some steps carried out by the systemto obtain a signal for analysis to characterize at least one periodiccomponent thereof in a first variant;

FIG. 5 is a flow chart illustrating some steps in carried out by thesystem to obtain a signal for analysis to characterize at least oneperiodic component thereof in a second variant; and

FIG. 6 is a flow chart illustrating an alternative method of obtaining afirst signal for analysis to characterize at least one periodiccomponent thereof.

A system for carrying out remote photoplethysmography is illustrated byway of example in FIG. 1. Photoplethysmographic imaging is based on theprinciple that temporal variations in blood volume under the skin leadto variations in light absorption by the skin. Such variations can bedetected and measured by taking images of an area of skin andcalculating the pixel average over a selected region. They can also bedetected by illuminating a selected area of skin with light andmeasuring the intensity of reflected light with a photosensor. In theexample used herein, the system of FIG. 1 is arranged to determine thevalue of the heart rate of a living being. However, the same system canbe used to characterize the phase and/or frequency of some otherperiodic biological phenomenon, e.g. the respiration rate or oxygenationlevel.

It is noted that the system illustrated in FIG. 1 and the principles tobe explained below can also be used to acquire non-biological temporallyvibrating or pulsating signals from images. Examples of suitableapplications include the monitoring of industrial equipment andprocesses to predict impending failure by analyzing vibrations andremote detection of the number of revolutions per second of the enginein a motor vehicle to determine its speed. In all these examples, as inthe example to be explained in detail herein, a method is used that isrobust to changes in illumination and to motion unrelated to theperiodic phenomenon to be analyzed.

In the illustrated embodiment, the system includes a data processingsystem 1, which can be a general-purpose computer provided withappropriate interfaces or a dedicated device. The data processing system1 includes a data processing device 2 and main memory 3, as well as amass-storage device 4. It further includes at least an interface 5 to anoutput device 6.

The data processing system 1 of the example is arranged to acquire asequence of images from a video camera 7 via an interface 8. Moreexactly, the data processing system 1 is arranged to receive a pluralityof sequences of image frames in respective color channels. The imagesframes in a particular color channel comprise pixel valuesrepresentative of intensities of captured electromagnetic radiation in aparticular range of the electromagnetic spectrum. This is due to colorfilters comprised in the video camera 7.

The sequences of image frames in the different color channels aregenerally synchronized, in the sense that one image frame from eachchannel is associated with a particular point in time. If the videocamera 7 has only one photosensor array, there will be a slight shift intime between the associated image frames, but the shift will be an orderof magnitude smaller than that corresponding to the frequency with whichcomplete images are captured. Generally, however, the video camera 7will have multiple photosensor arrays, so that the image frames can becaptured simultaneously. In an alternative embodiment, multiple videocameras are used.

It is noted that the system can also use photosensors 9-11 withappropriate filters to obtain signals representative of intensities ofcaptured electromagnetic radiation, each corresponding to a respectivedifferent frequency range. The signals from the photosensors 9,10,11 arereceived by the data processing system 1 through an interface 12. In onevariant of such an embodiment, a first sensor 9 is arranged to capturered light, a second sensor 10 blue light and a third sensor 11 greenlight. In an alternative embodiment, one of the sensors 9-11 is arrangedto capture a signal (analogue or digital) comprising intensity values ofcaptured electromagnetic radiation in a part of the electromagneticradiation spectrum corresponding to a wavelength in air in only a rangecomprised in a range between 500 nm and 600 nm. This corresponds to thegreen part of the visible spectrum.

In the same or another embodiment, one of the sensors 9-11 is tuned to afrequency corresponding to a peak in the absorption spectrum of water.Suitable frequencies in the visible part of the spectrum are thosecorresponding to a wavelength in air of 514, 606, 660 or 739 nm. In analternative variant, one of the sensors 9-11 is tuned to a frequencyjust outside the visible part of the spectrum (that is in the nearinfra-red). In particular, the sensor can be tuned to a frequencycorresponding to a wavelength in air of 836 nm or 970 nm. Siliconphotosensors are particularly sensitive to radiation in the rangebetween 800 and 1000 nm, so that such an embodiment results in a strongcaptured signal. Silicon photosensors are relatively cheap and common.Although both the video camera 7 and the photosensors 9-11 are shown inFIG. 1, it will be understood that the system can generally use only oneof these modalities for capturing signals representative of intensitiesof captured electromagnetic radiation corresponding to differentrespective radiation frequency ranges. The present description willproceed on the basis of the embodiment using the video camera 7.

In one embodiment, a standard video camera 7 with Red, Green and Bluechannels is used. In another embodiment, the channels can be Cyan,Yellow and Magenta.

One embodiment uses an adapted video camera 7 with a filter that admitslight within only a limited range comprised in the range of frequenciescorresponding to wavelengths in air of between 500 nm and 600 nm. Thismeans that image frames in this channel comprise pixel valuescorresponding to captured intensities of green light.

Another embodiment comprises an adapted video camera 7 with a filtertuned to a peak in the absorption spectrum of water. In particular, thefilter may be tuned to a frequency corresponding to a wavelength in airof 514, 606, 660 or 739 nm. Where it is desirable to use the system ofFIG. 1 in relatively dark situations, the filter can be tuned to afrequency just outside the visible part of the spectrum (that is in thenear infra-red). In particular, the sensor can be tuned to a frequencycorresponding to a wavelength in air of 836 nm or 970 nm. This has theeffect that a video camera 7 with a conventional CMOS or CCD array canbe used. Such sensors are particularly sensitive to radiation in therange between 800 and 1000 nm, so that such an embodiment results in astrong captured signal. When used to capture images of one or moreliving beings, the pixel values in this channel vary relatively stronglywith blood plasma flow. It is thus possible to extract a first signalwith a relatively strong component having a frequency corresponding theliving being's heart rate, as will be explained.

In the illustrated embodiment, the system of FIG. 1 is provided with an(optional) set of light sources 13-15, each tuned to a particularfrequency range. In an embodiment, the light sources 13-15 are tuned tofrequencies or frequency ranges corresponding to those of the channelsof the video camera 7. The data processing system 1 is configured tocontrol the intensities of light emitted in the respective frequencyranges by means of an appropriate interface 16. In an furtherembodiment, the data processing system 1 the light sources 13-15 aretunable, and the data processing system 1 is arranged to selectappropriate parameter values controlling the range of theelectromagnetic spectrum within which the light sources 13-15 emitlight. This can be combined with appropriate tunable filters for thevideo camera 7.

FIG. 2 shows steps in a first method carried out by the data processingsystem 1 that will be explained also with reference to FIG. 3. In afirst step 17, the data processing system 1 loads sequences 18-20 ofimage frames from each channel. The sequences 18-20 are synchronized inthe sense that a particular point in time corresponds to one image framefrom each sequence 18-20. Each sequence thus forms a signal, to which atransformation is applied in order to obtain a single output signal. Thetransformation is of such a nature that each value of the output signalis based on values from each respective second signal at correspondingpoints in time.

That said, the transformation comprises multiple operations, some ofwhich only take an image frame from one of the sequences 18-20 as input.In the illustrated embodiment, the method includes a step 21 of carryingout an operation 22 a-c (FIG. 3) in parallel on one frame from each ofthe respective sequences 18-20. The operation 22 a-c is a parameterizedoperation 22 a-c, meaning that it takes a parameter value as a furtherinput, which parameter value is set separately for each of therespective channels. Generally, it will therefore be different for eachof the respective channels. The operation 22 a-c can be a non-linearoperation, as is illustrated in FIG. 3. An example of several suitableoperations is a gamma encoding operation (V_(out)=V_(in) ^(γ)), with aseparate gamma value being used for each channel. The output of thisstep 21 comprises three parallel sequences 23-25 of individuallyprocessed image frames.

A next operation 26 is carried out (step 27) on the sequences 23-25provided as output in the previous step 21. This operation 26 is anoperation taking at least two inputs based in different ways on thesequences 23-25 of individually processed image frames. In the example,the inputs correspond directly to the sequences 23-25. In the example,the operation 26 is a matrix operation. It transforms three image framesinto a single image (i.e. an array of values based on pixel values), sothat the sequences 23-25 of individually processed image frames aretransformed into a single image sequence 28. This step 27 is implementedsuch that image frames corresponding to a common point in time are usedas input.

The output of the matrix operation 26 is subsequently processed (step29) in a further non-linear operation 30. This can again be a gammaencoding operation. The result is an output sequence 31 of images (i.e.arrays of values based on pixel values). From the output sequence 31, atleast one signal is extracted (step 32) for analysis to characterize atleast one periodic component thereof. The extracted signal is in theform of a sequence of single values, as opposed to a sequence of arraysof values. Each value is based on multiple pixel values from one imagein the output sequence 31.

In an alternative implementation of the method illustrated in FIG. 3,the step 21 of carrying out an operation 22 a-c (FIG. 3) in parallel onone frame from each of the respective sequences 18-20 does not use agamma encoding operation. Rather, the operation 22 a-c comprises takingthe logarithm of the pixel values of the sequences 18-20. In thisalternative implementation, the operation 22 a-c need not necessarily beparameterized, but it is of course non-linear. Also in thisimplementation, the output of the step 21 of carrying out the non-linearoperation 22 a-c in parallel, frame-by-frame on the respective sequences18-20 comprises three parallel sequences 23-25 of individually processedimage frames.

Taking the logarithm is useful for removing signal components due tochanges in illumination affecting all the channels. Such changes resultfrom movement of the illuminated subject (e.g. resulting in a shadowfalling over an entire region of interest in the frames and/or fromchanges in background illumination). They are generally multiplicative,in the sense that the signal of interest in each of the channels ismultiplied by a modulating factor representative of brightness changesand any changes in the spectrum of the source of illumination. By takingthe logarithm, the background illumination becomes additive, making iteasier to remove in, for example, the subsequent matrix operation 26.The matrix operation 26 can, for example, involve a projection on aplane in color space that leads to the complete or partial removal ofthe modulating signal representative of luminance changes. Taking thelogarithm does not appreciably affect the signal of interest, i.e. thesignal to be extracted from the output sequence 31 of images, becausethe signal of interest, as will become clear has relatively smallvariations. Thus, the logarithm approximates to that signal, which willbe appreciated by looking at the Taylor expansion of the naturallogarithm (ln z=(z−1)−½(z−1)²+ . . . ).

In a further embodiment (not shown in detail in FIG. 2), the matrixoperation 26 is preceded by an operation in which the effects of certainvariations in conditions that affect all color channels in a generallyequal measure are reduced or eliminated. For example, in remotephotoplethysmography, ambient illumination changes, changes in skintransmission and imperfect tracking of regions of interest can lead tomodulations of the signals in each of the color channels in the sameway. This effect is reduced or removed by dividing each of correspondingpixel values of associated frames of the respective sequences 18-20 by avalue that is a linear combination of the pixel values from each of thesequences 18-20. It is noted that embodiments are possible in which allbut one of the coefficients of the linear combination are equal to zero.

One can write the sequences 18-20 of image frames in the differentrespective color channels as follows:

$\begin{matrix}{{{\overset{\rightarrow}{I}\left( {\overset{\rightarrow}{x},t} \right)} = \begin{bmatrix}{R_{c}\left( {\overset{\rightarrow}{x},t} \right)} \\{G_{c}\left( {\overset{\rightarrow}{x},t} \right)} \\{B_{c}\left( {\overset{\rightarrow}{x},t} \right)}\end{bmatrix}},} & (1)\end{matrix}$

with {right arrow over (x)} being the location of a pixel in a frame,and t being a point in time corresponding to a frame in each respectivesequence 18-20. The removal or reduction of the effect of luminancevariations in the step 21 preceding the matrix operation 26 can then bewritten as follows:

$\begin{matrix}{{{\overset{\rightarrow}{I}}_{b}\left( {\overset{\rightarrow}{x},t} \right)} = {\frac{1}{{\alpha \; {R_{c}\left( {\overset{\rightarrow}{x},t} \right)}} + {\beta \; {G_{c}\left( {\overset{\rightarrow}{x},t} \right)}} + {\gamma \; {B_{c}\left( {\overset{\rightarrow}{x},t} \right)}}}{{\overset{\rightarrow}{I}\left( {\overset{\rightarrow}{x},t} \right)}.}}} & (2)\end{matrix}$

It is noted that this is a parameterized operation, and that the valuesof the coefficients α, β, γ can be optimized by analyzing at least oneof the signals represented by the sequences 18-20, the signal resultingfrom the matrix operation 26 or the signal extracted from the lattersignal. However, in general, any set of coefficient values will providean effect, including a combination in which two of these coefficientshave the value zero.

Implementations of the step 32 of extracting at least one signal foranalysis to characterize at least one periodic component thereof willnow be described.

A first example of an extraction method is illustrated in FIG. 4. Somesteps have been omitted for clarity. The order of steps can be varied toa certain extent and certain steps can be carried out at an earlierstage than shown. In particular, certain steps can be carried out at thestage when the image sequences 18,19,20 received from the video camera7. The illustrated method commences by obtaining (step 33) the outputsequence 31 resulting from the last non-linear processing step 29 shownin FIG. 2. In a next step 34, the images are processed to removenon-periodic background signals. In an implementation of this step 34, acorrection signal corresponding to the average brightness of part or allof the images in the sequence 31 is formed. The pixel data are thendecorrelated from the correction signal. Algorithms for cancelingnon-linear cross-correlations are known per se. Further image processingcan take place in this step 34, e.g. to compensate for camera motion.

In the method of FIG. 4, a region of interest 35 is tracked (step 37)through the sequence 31 of images. The region of interest 35 is selectedto correspond to a part of the images representing exposed skin of partof a living being, e.g. the face. A suitable algorithm for selectingsuch a region of interest 35 is described e.g. in Viola, P. and Jones,M. J., “Robust real-time object detection”, Proc. Of IEEE Workshop onstatistical and computational theories of vision, 13 Jul. 2001. Othertechniques can be used instead of or in combination with this technique.

The region of interest 35 is tracked throughout the sequence 31. Asuitable tracking algorithm is described in De Haan et al., “True-motionestimation with 3-D recursive search block matching”, IEEE Transactionson circuits and systems for video technology, 3 (5), October 1993, pp.368-379. Other tracking algorithms can be used alternatively or incombination with this tracking algorithm.

In the illustrated embodiment, a measurement zone 36 is selected (step38) within the region of interest 35. The measurement zone 36 isselected by means of a spatial and/or temporal analysis of a pluralityof image parts, each one or more image points in size, to determine aset of contiguous parts having similar characteristics. A suitablealgorithm is an algorithm for selecting a region with minimal gradientvariations. The position of the measurement zone 36 is determinedrelative to the region of interest 35 in a key image, so that it cansubsequently be located in each of the images of the sequence 31.

Next (step 39), values associated with individual image points withinthe measurement zone 36 are combined for each image, so that abrightness signal 40 is obtained. A suitable combination operation isaveraging. This signal 40 is a time-varying signal, since each valuecorresponds to an image in the sequence 31 and each image in thesequence 31 corresponds to a particular point in time.

This brightness signal 40 is then converted (step 41) into a finalsignal 42 representative of at least variations in a value based onpixel values from the sequences 18-20 obtained from the video camera 7.In an implementation of this step 41, the brightness signal 40 iscentered on its mean value. In a different implementation, this step 41comprises a filtering operation, e.g. a filtering operationcorresponding to differentiation of the average brightness signal 40.Other alternative techniques for extracting variations of the order of1% of the dynamic range of the brightness signal 40 are also possible.

The final signal 42 is analyzed (step 43) to characterize one or moreperiodic biological phenomena. In particular, a spectral analysis isused to determine one or more local maxima in its spectrum, at leastwithin a range of frequencies known to comprise typical heart ratevalues and/or respiration rate values for the living beings of interest(e.g. human beings). In an alternative embodiment, a heart rateextraction method in the time domain is used.

It is observed that a single region of interest 35 and a singlemeasurement zone 36 have been described for illustrative purposes. Inother embodiments, multiple regions of interest within the sequence 31are tracked and/or multiple measurement zones within one region ofinterest are used to obtain multiple final signals. Clusteringalgorithms or the like can then be used to arrive at a consensus valuefor the heart rate, for example.

FIG. 5 illustrates an alternative method of processing the sequence 31of images obtained using the method of FIG. 2. Again, certain steps havebeen omitted for clarity, the order of steps can be changed and certainsteps can already be carried out at the stage illustrated in FIG. 2.

Initial steps 44,45 are similar to the first steps 33,34 of the methodof FIG. 4. Thus, the sequence 31 of images generated using the method ofFIG. 2 is obtained (step 44) and corrected (step 45).

However, in a next step 46, a grid is laid over each of the images inthe sequence 31. The grid partitions the images into an array of(potential) measurement zones. Each measurement zone includes aplurality of image points, i.e. pixel locations. Then (step 47), atleast one, but generally all of the measurement zones are selected, and,for each measurement zone, a respective signal 48 a-n corresponding tothe time-varying spatial combination of pixel values at image points inthe measurement zone concerned is formed. The spatial combination can bean average, mean value or other type of combination. In any case, thevalue of one of the signals 48 a-n at a particular point in time isbased on multiple pixel values from the image in the sequence 31corresponding to that point in time. Thus, random noise (as opposed tonoise due to camera motion or movement of objects in the representedscene) is cancelled.

Each of the signals 48 a-n is then centered on its mean value (step 49)to yield a final signal 50 a-n for analysis to characterize at least oneperiodic component thereof. Again, instead of centering the signals 48a-n on their mean value, a different operation for extractingsmall-amplitude variations, such as differentiation, can be used.

The analysis of the final signals 50 a-n is carried out in a last step51. In this example, the dominant frequency in a particular frequencyrange is determined. In other applications, a phase map is created for aparticular frequency (if a final signal 50 a-n is created for eachmeasurement zone defined by the grid).

The methods described above with reference to FIGS. 2, 4 and 5 allow forrobust continuous monitoring of biometrical signals of people. Moreover,there is minimal interference with the monitored person's dailyactivities, due to the remote nature of the monitoring. The method canthus be used to provide virtually immediate feedback on a person'smental and physical condition at any instance in time. The method doesnot require the person being monitored to wear annoying body sensors.

Conventional remote photoplethysmographic methods are limited to remotemonitoring of persons within a range of about 1 meter of the videocamera 7. Moreover, the subject being monitored should not move, and theillumination should be constant during the acquisition of the videosignal.

Steps 52,53 illustrated in FIG. 2 enable the system of FIG. 1 to providefinal signals 42,50 a-n with increased signal-to-noise ratios, at leastin as far as the components corresponding to the periodic signals to beanalyzed (the components indicative of pulsating blood flow in theexample of remote photoplethysmography) are concerned. In particular,the robustness to noise introduced by movement of the video camera 7,movement of the person being monitored and variations in illumination isimproved.

To this end, the system analyses (step 52) at least one of the sequences18-20 of image frames received from the video camera 7, the sequences23-25 of individually processed image frames, the image sequence 28resulting from the matrix operation 26, the output sequence 31 ofimages, the extracted signal 40,48 and the signal 42,50 a-n, anddetermines (step 53) appropriate values to provide a set 54 of parametervalues for each instance of the first operation 22 a-c, the matrixoperation 26 and the further non-linear transformation operation 30.Alternatively or additionally, parameters for deriving values of controlsignals determining the relative intensities and/or frequencies of lightemitted by the different tuned light sources 13-15 and/or filtercoefficients determining the relative attenuations and/or or pass bandof light captured by the sensors 9-11 can be included in the set 54 ofparameter values.

The parameter values are chosen to optimize the ratio of the wantedsignal component (representative of blood volume pulsations) to unwantedsignal components (due to illumination changes or motion).

In one embodiment, the data processing system actually calculates thesignal-to-noise ratio using the final signals 42,50 a-n and adjusts theparameter values using an exhaustive or directed search method so as tomaximize the calculated signal-to-noise-ratio.

In another embodiment, the data processing system 1 carries out aprincipal component analysis using only the sequences 18-20 of imageframes in the color channels or the individually processed sequences23-25. Coefficients for the matrix operation are then chosen to beorthogonal to the principal components that are unrelated to the signalcomponent of interest, i.e. the component representative of the heartrate and/or respiration rate.

In the illustrated embodiment, the steps 52,53 are carried outcontinually whilst the signals represented by the sequences 18-20 arebeing processed. This embodiment is suitable for situations in whichambient lighting condition change.

In an alternative embodiment, a system as shown in FIG. 1 is used toobtain a set 54 of parameter values for subsequent use in one or moreother systems that carry out only the steps 17,21,27,29,32 that resultdirectly in the signals 42,50 a-n for analysis to characterize at leastone periodic component thereof. Thus, a method of manufacturing suchother systems includes configuring the systems with the set 54 ofparameter values previously obtained using test signals and the fullmethod of FIG. 2. Those systems, like the system carrying out the fullmethod of FIG. 2, are able to extract a signal including a componentcarrying information representative of a periodic signal that isrelatively strong.

A further embodiment of a method of carrying out remotephotoplethysmography is illustrated in FIG. 6. This method can beregarded as an alternative to the method of FIG. 2, and is likewisecarried out by the data processing system 1.

This method also takes as input a plurality of sequences 55-57 of imageframes in respective color channels, corresponding to the sequences18-20 illustrated in FIG. 2 and discussed above. Each image frame iscomprised of pixel values representative of intensities of capturedelectromagnetic radiation in a frequency range associated with thesequence 55-57 concerned, a different respective frequency range beingassociated with each of the sequences 55-57. The sequences 55-57 aresynchronized, in that a particular point in time corresponds to oneimage frame of each sequence 55-57. Each sequence 55-57 thus forms asignal, also referred to as a second signal herein. The image framescorrespond to images of at least one living being, preferably a humanbeing, with exposed skin. In the following detailed explanation, it willbe assumed that complete image frames are processed, but the method ofFIG. 6 can also be carried out on one or more tracked regions ofinterest, each the result of a segmentation operation aimed atidentifying parts of the image frames representing exposed skin.

The sequences 55-57 of images frames are obtained in a first step 58,after which they are processed in parallel. That is to say that eachforms an input to a respective sequence of at least one operation. Atleast one operation in each sequence involves the application of afactor of which the value is based on at least one value calculated onthe basis of pixel values from the images frames in the sequences 55-57themselves. These factors determine the relative influences of at leastcomponents of the sequences 55-57 on a final signal that is used toextract heart rate or respiration rate information (in aphotoplethysmography application).

In the illustrated embodiment, color normalization (step 59) is carriedout first, but the order of the operations in the sequence of operationscan in principle be different. The color normalization step 59 iscarried out to normalize the average skin tone within a time interval,and thus to eliminate the effects of slow changes in the illuminationspectrum and of slow changes in the average skin tone due to motion.Motion can lead to changes in the average skin tone when pixelsrepresenting skin are imperfectly tracked. It can also lead to changesin the average skin tone when the environment in which the depictedsubject moves is not illuminated uniformly, e.g. due to the presence ofmultiple light sources with different spectra or to colored reflectivesurfaces.

The color normalization step 59 in effect transforms the color spacedefined by the camera 7 to a normalized color space in which slow colorchanges have no effect. An effect of this normalization is (ideally) toprovide a vector in a normalized color space that moves in parallel to afixed line representative of color changes due to blood volumepulsations (the “heartbeat line”).

Assuming that the sequences 55-57 of image frames correspond to Red,Green and Blue color channels, one can write the color change {rightarrow over (H)}_(c)({right arrow over (x)},t) registered by the camera 7at a pixel position {right arrow over (x)} and point in time t asfollows:

$\begin{matrix}{{{\overset{\rightarrow}{H}}_{c}\left( {\overset{\rightarrow}{x},t} \right)} = {\frac{{{\overset{\rightarrow}{I}}_{c}\left( {\overset{\rightarrow}{x},t} \right)}}{t} = {\frac{}{t}\begin{bmatrix}{R_{c}\left( {\overset{\rightarrow}{x},t} \right)} \\{G_{c}\left( {\overset{\rightarrow}{x},t} \right)} \\{B_{c}\left( {\overset{\rightarrow}{x},t} \right)}\end{bmatrix}}}} & (3)\end{matrix}$

The change in color due to varying blood volume (the phenomenon ofinterest in the case of remote photoplethysmography) is modulated by thelocal skin tone of the skin covering the blood volume and by theillumination sources. This is a multiplicative process, so thatappropriate normalization will lead to demodulation.

The color normalization step 59 includes dividing the instantaneouscolor components by the corresponding time-average of the red, green andblue values. Instead of the average, a different combination of multiplevalues associated with respective different points in time spanning thepoint in time associated with the point in time for which thecombination is being calculated could be taken. An example is the medianvalue. The result of the color normalization step 59 is formed by threeparallel sequences 60-62 of color-normalized images. The operation canbe written as follows:

$\begin{matrix}{{{{\overset{\rightarrow}{I}}_{n}\left( {\overset{\rightarrow}{x},t} \right)} = {\begin{bmatrix}{R_{n}\left( {\overset{\rightarrow}{x},t} \right)} \\{G_{n}\left( {\overset{\rightarrow}{x},t} \right)} \\{B_{n}\left( {\overset{\rightarrow}{x},t} \right)}\end{bmatrix} = \begin{bmatrix}\frac{R_{c}\left( {\overset{\rightarrow}{x},t} \right)}{{\overset{\_}{R}}_{c}(t)} \\\frac{G_{c}\left( {\overset{\rightarrow}{x},t} \right)}{{\overset{\_}{G}}_{c}(t)} \\\frac{B_{c}\left( {\overset{\rightarrow}{x},t} \right)}{{\overset{\_}{B}}_{c}(t)}\end{bmatrix}}},} & (4)\end{matrix}$

wherein {right arrow over (I)}_(n)({right arrow over (x)},t) is anormalized color vector, and R _(c)(t), G _(c)(t), B _(c)(t) aretime-varying factors calculated separately for each of the parallelsequences 55-57 of image frames.

In one alternative variant to the one discussed herein, separate factorvalues R _(c)({right arrow over (x)},t), G _(c)({right arrow over(x)},t), B({right arrow over (x)},t) are calculated and applied for eachpixel position (i.e. there is no spatial combination).

In the illustrated variant, the factors R _(c)(t), G _(c)(t), B _(c)(t)are calculated on the basis of pixel values at multiple pixel positionswithin each of one or more regions of interest, and applied to pixelsassociated with positions within this region of interest. Thus, they canbe written as follows:

$\begin{matrix}{{{{{\overset{\_}{R}}_{c}(t)} = {\frac{1}{2ɛ{X}}{\int_{t - ɛ}^{t + ɛ}{\int_{X}{{R_{c}\left( {\overset{\rightarrow}{x},z} \right)}\ {\overset{\rightarrow}{x}}\ {z}}}}}},{{{\overset{\_}{G}}_{c}(t)} = {\frac{1}{2ɛ{X}}{\int_{t - ɛ}^{t + ɛ}{\int_{X}{{G_{c}\left( {\overset{\rightarrow}{x},z} \right)}\ {\overset{\rightarrow}{x}}\ {z}}}}}},{and}}{{{{\overset{\_}{B}}_{c}(t)} = {\frac{1}{2ɛ{X}}{\int_{t - ɛ}^{t + ɛ}{\int_{X}{{B_{c}\left( {\overset{\rightarrow}{x},z} \right)}\ {\overset{\rightarrow}{x}}\ {z}}}}}},}} & (5)\end{matrix}$

wherein |X| corresponds to the area of the region of interest X.

Of course, the data processing device 2 will process time-discretesignals. Although the average over time can use pixel values from morethan two points in time, a simple and relatively accurate approximationuses pixel values from two successive points in time. One of these cancorrespond to the point in time for which the factor value is beingcalculated. In one example of an implementation of such a step, this canbe written as:

$\begin{matrix}{{I_{n,d}\left( {x,t} \right)} = {\frac{I_{c,d}\left( {\overset{\rightarrow}{x},t} \right)}{{I_{c,d}\left( {\overset{\rightarrow}{x},t} \right)} + {I_{c,d}\left( {\overset{\rightarrow}{x},{t - 1}} \right)}}.}} & (6)\end{matrix}$

Note that the region of interest corresponds to one single pixelposition in this variant.

The color normalization step 59 should result in image frames of whichthe values of the pixels obey a Gaussian distribution. In a variant,those pixels with outlying values are not used to obtain a final signal(also referred to as “first signal” herein) for analysis to characteriseat least one periodic component thereof. Additionally or alternatively,the method is applied iteratively. After color normalization, a regionwith the most stable distribution of pixel values over time is used asregion of interest. Alternatively, the largest region could be used. Inboth alternatives, the fact that image parts representing exposed skinof a single living being should have a Gaussian distribution is used tosegment the image frames. A color normalization factor appropriate toone of these segments is calculated and applied to pixel valuesassociated with pixel positions in only that segment.

As illustrated in FIG. 6, the sequences 60-62 of color-normalized imagesundergo further processing in a next step 63, in which illuminationnormalization is carried out. It is noted that this step 63 can precedethe color normalization step 59, and that one of these two normalizationsteps 59,63 can be omitted in some variants.

The illumination normalization step 63 takes account of the nearimpossibility of accurately tracking pixels through the sequences 55-57of images, although it should generally be possible to segment out allthose pixels representing exposed skin. However, variations inillumination strength and transmission of the skin will still show upwhen individual pixels are not tracked perfectly. Illuminationnormalization helps remove these effects. It also helps remove changesin brightness that are due to movement, rather than pulsating blood flowof the subject shown in the images.

The illumination normalization step 63 results in three furthernormalized sequences 64-66 of images. It can be representedmathematically as follows:

$\begin{matrix}{{{\overset{\rightarrow}{I}}_{b}\left( {\overset{\rightarrow}{x},t} \right)} = {\frac{1}{{\alpha \; {R_{n}\left( {\overset{\rightarrow}{x},t} \right)}} + {\beta \; {G_{n}\left( {\overset{\rightarrow}{x},t} \right)}} + {\gamma \; {B_{n}\left( {\overset{\rightarrow}{x},t} \right)}}}{{\overset{\rightarrow}{I}}_{n}\left( {\overset{\rightarrow}{x},t} \right)}}} & (7)\end{matrix}$

The parameters α, β, γ can have any value. Indeed, one or two of themcan have the value zero. However, there is theoretically an optimumchoice of values, which is that combination that causes the resultingnormalized pixels to lie in a plane parallel to the heartbeat line inthe normalized color space. In one embodiment, an analysis is carriedout in order to select optimized values of the parameters α, β, γ.

In a subsequent step 67 of the illustrated method, difference signalsare established by subtracting from each current pixel value the valueat the corresponding pixel position at the immediately preceding pointin time, resulting in three sequences 68-70 of frames of differencevalues. It is noted that this step 67 can be combined with, for example,the color-normalization step 59, so that Equation (6) is modified to:

$\begin{matrix}{{H_{n,d}\left( {x,t} \right)} = \frac{{I_{c,d}\left( {\overset{\rightarrow}{x},t} \right)} - {I_{c,d}\left( {\overset{\rightarrow}{x},{t - 1}} \right)}}{{I_{c,d}\left( {\overset{\rightarrow}{x},t} \right)} + {I_{c,d}\left( {\overset{\rightarrow}{x},{t - 1}} \right)}}} & (8)\end{matrix}$

The illustrated step 67 is the last in the parallel sequences ofoperations taking the sequences 55-57 of image frames (the “secondsignals”) as input.

There follows a further step 71, in which an operation is carried outthat combines normalized difference signals into a single signal ofwhich each value is based on values from the respective normalizeddifference signals at corresponding points in time. In the illustratedembodiment, this step 71 involves a projection onto a line in thenormalized color space that corresponds to color variations due to thepulsating blood flow of a living being. Any noise orthogonal to thisline is eliminated in this step 71, which results in a single sequence72 of frames. In a variant, outliers are also eliminated in this step71.

Then, an optional post-processing step 73 is applied. This step 73 caninvolve such operations as bandpass filtering. The result is a sequence74 of post-processed image frames.

Finally (step 75), at least one combination signal 76 is established.This signal is based on signals at multiple pixel positions, the resultof processing them to obtain a consensus value. This step 75 can involveclustering, averaging, determining a median value, further outlierremoval, and the like. This step 75 results in a stronger signal with abetter signal-to-noise ratio. Fourier analysis can be applied tocharacterize at least one periodic component, e.g. the periodiccomponent corresponding to the heartbeat signal or respiration signal.

In an alternative embodiment, the post-processing step 73 alreadyinvolves a transformation from the time domain to the frequency domain.

The effect of the embodiments described above is to enable imagescaptured with a relatively uncomplicated and inexpensive camera 7 to beused to characterize periodic phenomena that results in almostimperceptible colour variations in regions of interest within the imageframes. The characterization is relatively reliable, because variationsdue to other influences are eliminated or at least suppressed.

It should be noted that the above-mentioned embodiments illustrate,rather than limit, the invention, and that those skilled in the art willbe able to design many alternative embodiments without departing fromthe scope of the appended claims. In the claims, any reference signsplaced between parentheses shall not be construed as limiting the claim.The word “comprising” does not exclude the presence of elements or stepsother than those listed in a claim. The word “a” or an preceding anelement does not exclude the presence of a plurality of such elements.The mere fact that certain measures are recited in mutually differentdependent claims does not indicate that a combination of these measurescannot be used to advantage.

An embodiment in which both the video camera 7 and the sensors 9-11 areused is also possible, for example. Another embodiment in which afurther camera is used in addition to the video camera 7 is alsopossible. For example, a thermal camera can provide a signal in onechannel with the video camera 7 providing a signal in another channel.

1-15. (canceled)
 16. A system for characterizing periodic component of afirst signal, the system comprising a processor configured to: obtainsecond signals representative of intensities of captured electromagneticradiation, each corresponding to a respective different radiationfrequency range; analyze the second signals by applying a transformationto the second signals to form transformed signals such that any value ofthe transformed signals is based on values from each of the secondsignals at corresponding points in time; combine the transformed signalsinto an output signal; and obtain values of variables determininginfluences of components of the second signals on the output signal byat least one of: select at least one value of at least one parametercorresponding to a respective one of the variables; and calculate valuesof at least one time-varying factor corresponding to a respective one ofthe variables, and perform a normalization of the second signals usingthe calculated values.
 17. The system of claim 16, wherein the processoris configured to use analysis of at least one of the second signals, theoutput signal and the first signal to select at least one value of atleast one parameter determining influences of at least components ofrespective ones of the second signals on the output signal so as tomaximize a relative signal strength in a limited part of a spectrum ofthe first signal.
 18. The system of claim 16, wherein the transformationincludes at least one parameterized operation, and wherein the processoris configured to use analysis of at least one of the second signals, theoutput signal and the first signal to select values of at least oneparameter of the operation.
 19. The system of claim 18, wherein theparameterized operation is an operation taking at least two inputs basedin different ways on the second signals.
 20. The system of claim 18,wherein the transformation includes an operation carried out in parallelon respective input signals using at least one parameter value specificto fewer than all of the input signals.
 21. The system of claim 18,wherein the transformation includes at least one parameterizednon-linear operation, and wherein the processor is configured to useanalysis of at least one of the second signals, an output signalobtained by applying the transformation to the second signals and afurther signal derived from the output signal to select values of atleast one parameter of the parameterized operation.
 22. The system ofclaim 16, wherein the transformation includes an operation carried outin parallel on respective input signals, each a different linearcombination of the second signals with at least one coefficientdiffering from zero, and wherein the operation comprises taking alogarithm of the input signal.
 23. The system of claim 16, wherein theprocessor is configured to analyze the second signals by carrying out aprincipal component analysis on data sets respectively based on thesecond signals.
 24. The system of claim 16, wherein the processor isconfigured to analyze at least one of the second signals, an outputsignal obtained by applying the transformation to the second signals andthe first signal derived from the output signal, and to use the analysisto select at least one value of at least one parameter determininginfluences of at least components of respective second signals on theoutput signal when signals corresponding to the second signals arecaptured and the transformation is applied, wherein the first signal isderivable from the output signal, wherein any value of the output signalis based on values from each respective second signal at a common pointin time, and wherein the processor is further configured to determineinfluences of respective second signals on the output signal at leastpartly in accordance with the selected values of the at least oneparameter.
 25. The system of claim 16, wherein the processor is furtherconfigured to continually analyze at least one of the second signals,the output signal and the first signal derived from the output signaland select the at least one value of the at least one parameter, whileprocessing the second signals to obtain the first signal.
 26. The systemof claim 16, wherein the processor is further configured to: calculatevalues of at least one time-varying factor corresponding to a respectiveone of the variables, each factor value based on at least one secondsignal value, and apply each factor in an operation in at least one of anumber of parallel sequences of operations comprising at least one suchoperation and taking a signal corresponding to a respective one of thesecond signals as input, wherein the at least one time-varying factorincludes an obtainable factor obtainable by calculating a linearcombination of corresponding values from each of at least one of thesecond signals, and wherein a same value of the obtainable factor isused in each of the parallel sequences of operations.
 27. The system ofclaim 16, wherein the processor is further configured to: calculatevalues of at least one time-varying factor corresponding to a respectiveone of the variables, each factor value based on at least one secondsignal value, and apply each factor in an operation in at least one of anumber of parallel sequences of operations comprising at least one suchoperation and taking a signal corresponding to a respective one of thesecond signals as input, and wherein, for at least one of the factors,separate values at corresponding associated points in time arecalculated for each of the parallel sequences, each obtained frommultiple values of the second signal taken as input by the sequenceconcerned, the multiple values being associated with respective pointsin time spanning an interval including the point in time associated withthe factor value.
 28. A non-transitory computer readable mediumcomprising computer instructions which, when executed by a processor,configure the processor to perform acts of: obtaining two signalsrepresentative of intensities of captured electromagnetic radiation,each corresponding to a respective different radiation frequency range;analyzing the two signals by applying a transformation to the twosignals to form transformed signals such that any value of thetransformed signals is based on values from each of the two signals atcorresponding points in time; combining the transformed signals into anoutput signal; and obtaining values of variables determining influencesof components of the two signals on the output signal by at least oneof: selecting at least one value of at least one parameter correspondingto a respective one of the variables; and calculating values of at leastone time-varying factor corresponding to a respective one of thevariables, and performing a normalization of the two signals using thecalculated values.
 29. The non-transitory computer readable medium ofclaim 28, wherein the instructions, when executed by the processor,further configure the processor to use analysis of at least one of thesecond signals, the output signal and the first signal to select atleast one value of at least one parameter determining influences of atleast components of respective ones of the second signals on the outputsignal so as to maximize a relative signal strength in a limited part ofa spectrum of the first signal.
 30. The non-transitory computer readablemedium of claim 28, wherein the transformation includes at least oneparameterized operation, and wherein the instructions, when executed bythe processor, further configure the processor to use analysis of atleast one of the second signals, the output signal and the first signalto select values of at least one parameter of the operation.
 31. Thenon-transitory computer readable medium of claim 30, wherein theparameterized operation is an operation taking at least two inputs basedin different ways on the second signals.
 32. The non-transitory computerreadable medium of claim 30, wherein the transformation includes anoperation carried out in parallel on respective input signals using atleast one parameter value specific to fewer than all of the inputsignals.
 33. The non-transitory computer readable medium of claim 30,wherein the transformation includes at least one parameterizednon-linear operation, and wherein the instructions, when executed by theprocessor, further configure the processor to use analysis of at leastone of the second signals, an output signal obtained by applying thetransformation to the second signals and a further signal derived fromthe output signal to select values of at least one parameter of theparameterized operation.
 34. The non-transitory computer readable mediumof claim 28, wherein the transformation includes an operation carriedout in parallel on respective input signals, each a different linearcombination of the second signals with at least one coefficientdiffering from zero, and wherein the operation comprises taking alogarithm of the input signal.
 35. The non-transitory computer readablemedium of claim 28, wherein the instructions, when executed by theprocessor, further configure the processor to analyze the second signalsby carrying out a principal component analysis on data sets respectivelybased on the second signals.
 36. The non-transitory computer readablemedium of claim 28, wherein the instructions, when executed by theprocessor, further configure the processor to analyze at least one ofthe second signals, an output signal obtained by applying thetransformation to the second signals and the first signal derived fromthe output signal, and to use the analysis to select at least one valueof at least one parameter determining influences of at least componentsof respective second signals on the output signal when signalscorresponding to the second signals are captured and the transformationis applied, wherein the first signal is derivable from the outputsignal, wherein any value of the output signal is based on values fromeach respective second signal at a common point in time, and wherein theprocessor is further configured to determine influences of respectivesecond signals on the output signal at least partly in accordance withthe selected values of the at least one parameter.
 37. Thenon-transitory computer readable medium of claim 28, wherein theinstructions, when executed by the processor, further configure theprocessor to continually analyze at least one of the second signals, theoutput signal and the first signal derived from the output signal andselect the at least one value of the at least one parameter, whileprocessing the second signals to obtain the first signal.
 38. Thenon-transitory computer readable medium of claim 28, wherein theinstructions, when executed by the processor, further configure theprocessor to: calculate values of at least one time-varying factorcorresponding to a respective one of the variables, each factor valuebased on at least one second signal value, and apply each factor in anoperation in at least one of a number of parallel sequences ofoperations comprising at least one such operation and taking a signalcorresponding to a respective one of the second signals as input,wherein the at least one time-varying factor includes an obtainablefactor obtainable by calculating a linear combination of correspondingvalues from each of at least one of the second signals, and wherein asame value of the obtainable factor is used in each of the parallelsequences of operations.
 39. The non-transitory computer readable mediumof claim 28, wherein the instructions, when executed by the processor,further configure the processor to: calculate values of at least onetime-varying factor corresponding to a respective one of the variables,each factor value based on at least one second signal value, and applyeach factor in an operation in at least one of a number of parallelsequences of operations comprising at least one such operation andtaking a signal corresponding to a respective one of the second signalsas input, and wherein, for at least one of the factors, separate valuesat corresponding associated points in time are calculated for each ofthe parallel sequences, each obtained from multiple values of the secondsignal taken as input by the sequence concerned, the multiple valuesbeing associated with respective points in time spanning an intervalincluding the point in time associated with the factor value.